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SPECIFICATION 

TO ALL WHOM IT MAY CONCERN: 

BE IT KNOWN THAT WE, Osamu MIZUNO, a citizen 
of Japan residing at Setagaya-ku, Tokyo, Japan, 
Shinichi SAKAIDA, a citizen of Japan residing at 
Setagaya-ku, Tokyo,' Japan and Yoshiaki SHISHIKUI, a 
citizen of Japan residing at Setagaya-ku, Tokyo, Japan 
have invented certain new and useful improvements in 

APPARATUS AND METHOD FOR CODING BINARY IMAGE 
WITH IMPROVED EFFICIENCY 

of which the following is a specification:- 



EL698181592US 

-1- 



TITLE OF THE INVENTION 

APPARATUS AND METHOD FOR CODING BINARY 
IMAGE WITH IMPROVED EFFICIENCY 



5 BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

The present invention generally relates to 
object-shape coding apparatuses for coding a binary 
image representing an object shape by the unit of 

10 one rectangular block where the binary image is 

divided into a plurality of rectangular blocks, and 
particularly relates to an object-shape coding 
apparatus for coding rectangular blocks which 
includes both the pixels of the interior of the 

15 object shape and the pixels of the exterior of the 
object shape. 

2 . Description of the Related Art 

In recent years, interest has been high in 
object-based coding schemes such as ISO/IEC 14496-2: 

20 "Information Technology-Generic Coding of Audio- 
Visual Obj ects-Part2 : Visual.*' The object-based 
coding divides an original image into the images of 
objects such as people or the like in the foreground 
and objects in the background, and attends to image 

25 coding with respect to each object image separately. 
The object-based coding can achieve a higher coding 
efficiency than coding schemes based on the coding 
of image frame units such as the MPEG-2 video coding 
standard (ISO/IEC 13818-2: "Information Technology- 

30 Generic Coding of Moving Pictures and Associated 

Audio Information: Video"). Further, use of object- 
based coding provides a basis for making of a video 
by combining objects. 
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An object image is comprised of texture 
images and object-shape data. In the object-base 
coding, therefore, both the texture coding and the 
shape coding are performed. Shape data includes 
5 binary data of shape information that only 

represents shape, and further includes multi-level 
data of shape information that represents object 
transparency. The present invention relates to the 
binary data of shape information. 

10 In the following, related-art methods for 

binary shape coding will be described. 

There are two types of methods for 
representing object shapes. One is to use a bit 
pattern image that has binary values representing 

15 whether pixels are inside or outside the object 

boundary, and the other is to show only the object 
boundaries. Accordingly, object-based coding 
apparatuses can also be classified into two groups, 
one for coding binary bit pattern images and the 

20 other for coding contour data* 

Methods for coding binary bit pattern 
images attend to binary information coding by 
following the order of image scanning. Typical 
coding methods include the JBIG standard (ISO/IEC 

25 11544: "Progressive Bi~level Compression") and the 
MMR (modified modified read) coding standard (ITU-T 
T.6: "Facsimile Coding Schemes and Coding Control 
Functions for Group 4 Facsimile Apparatus"). The 
JBIG standard encodes binary data in a hierarchical 

30 manner by following the order of image scanning. 
The MMR standard encodes positions where binary 
pixels undergo changes in values, which is performed 
by following the order of image scanning. Both of 
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these two coding methods are loss-less processes. 

Methods for coding contour information 
attends to coding by following the order of points 
that make up the contour. Such methods include one 
5 that encodes directions of points that constitute 

the contour, and include one that reversibly encodes 
the coordinates of points that constitutes the 
contour. Among these, a chain coding scheme (Makoto 
Nagao, "Digital Image Processing,'* Kindaikagaku , 

10 pp. 384-385, 1987) assigns integers 1 through 8 to 
directions of connections relating the points that 
constitute the contour, and attends to reversible 
coding. Further, there is a method that carries out 
hierarchical coding by using the chain coding scheme 

15 (Tohru Kaneko, "Hierarchical Coding Scheme for Line 
Drawings Described by Chain Code Series," The 
Transactions of the Institute of Electronics, 
Information and Communication Engineers, Vol. J69-D, 
No. 5, 1986) , 

20 Further, methods for coding contour 

information include approximating for the contour by 
using the Spline function {Myron Flickner, et al . , 
"Periodic Quasi-Orthogonal Spline Bases and 
Applications to Least-Squares Curve Fitting of 

25 Digital Images," IEEE Transaction on Image 

Processing, vol. 5, No, 1, pp. 71-88, Jan, 1996), 
and also include a method using Wavelet descriptors 
(George Muller, et al . , "Progressive Transmission of 
Line Drawings Using the Wavelet Transform," IEEE 

30 Transaction on Image Processing, vol. 5, No. 4, pp. 
666-672, April 1996). Also included is a method 
that uses Wavelet descriptors for contour direction 
vectors (Japanese Patent Laid-open Application No. 
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All the binary shape coding methods as 
described above encode object shapes by the unit of 
one frame. 

5 In general, texture coding is conducted by 

the unit of one rectangular block after an original 
image is divided into a plurality of rectangular 
blocks. Among texture information within a given 
rectangular block, information is useful where it 

10 corresponds to the area of the object defined by the 
shape data. In order to keep consistency between 
the texture coding and the shape coding, some shape 
coding schemes employ division of an image into a 
plurality of rectangular blocks, and attend to 

15 block-specific coding. 

The binary shape coding of the MPEG-4 
standard divides a binary shape image into a 
plurality of rectangular blocks (macro blocks) of 16 
X 16 pixels where the binary shape image is 

20 comprised of shape interior pixels and shape 

exterior pixels, and attends to coding on the block- 
specific basis. The MPEG-4 standard is applicable 
to intra-frame coding as well as inter-frame coding. 
In the following, the intra-frame coding will be 

25 described. 

In the intra-frame coding, a coding mode 
is selected based on the conditions of the 
rectangular block, i.e., based on whether all the 
pixels of the rectangular block are those of the 
30 shape interior, whether all the pixels are those of 
the shape exterior, and whether the shape interior 
pixels and the shape exterior pixels are both 
present inside the rectangular block. When all the 
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pixels are shape interior pixels, or are shape 
exterior pixels, only the coding mode is transferred, 
without coding of each pixel. When the shape 
interior pixels and the shape exterior pixels are 
5 both present, a coded word is assigned to each pixel 
through arithmetic coding. 

The arithmetic coding is a type of a 
variable length coding scheme that reduces the 
quantity of information by utilizing disparity of 

10 symbol occurrence probabilities. In this coding 
scheme, a probability line segment is segmented 
according to the probabilities of occurrences of a 
symbol series, and a binary decimal value indicative 
of a position within a segmented section is used as 

15 a code for the symbol series (Hiroshi Harashima, 

"Image Information Compression,'* Ohm, pp. 153-161, 
1992.7). In the arithmetic coding, segmentation of 
a probability line based on probabilities of 
occurrences of a symbol series can be consecutively 

20 made through arithmetic operations, which achieves a 
compression efficiency that is close to the entropy 
limit of the symbol series. 

The Huffman coding is known as a variable 
length coding scheme that reduces the quantity of 

25 information by utilizing inequality of symbol 

occurrence probabilities in the same manner as in 
the arithmetic coding (Hiroshi Yasuda, Hiroshi 
Watanabe , "Basics of Digital Image Compression," 
Nikkei BP Publishing Center, pp. 32-35, 1996). In 

30 the Huffman coding, one coded word is assigned to 

one symbol . Since the Huffman coding only requires 
reading a coded word for a given symbol from the 
coded word table stored in memory, a coding 
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apparatus can be implemented as a small size 
apparatus . 

As described above, the MPEG-4 arithmetic 
coding has macro blocks of 16 x 16 pixels as input 
5 thereto, and attends to consecutive segmentation of 
a probability line segment for 256 pixel symbols. 
In general, coding efficiency increases as the 
processing block becomes bigger, but an increase in 
the processing block size entails needs for 

10 increased computation and increased memory. This is 
one of the factors that make it difficult to develop 
a real-time coding apparatus for an image of a large 
size such as an HDTV image. 

In order to reduce the computation load 

15 and the memory volume, input data may be coded by 
the unit of a small data size. Since real-time 
processing is performed by use of hardware, however, 
correlation within the data cannot be fully utilized 
if the coding is performed by the unit of a small 

20 data size. In order to obviate this problem, it is 
desirable to provide a coding apparatus that can 
achieve efficient coding while avoiding an increase 
in the size of hardware for code assigning process. 
Accordingly, there is a need for an 

25 object-shape coding apparatus that can achieve 

efficient coding while avoiding an increase in the 
size of hardware for code assigning processing where 
the object-shape coding apparatus divides a binary 
image representing an object shape into a plurality 

30 of rectangular blocks, and encodes each of the 
rectangular blocks separately, including a 
rectangular block which includes both object 
interior pixels and object exterior pixels. 
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SUMMARY OF THE INVENTION 

It is a general object of the present 
invention to provide a coding apparatus and a coding 
5 method that substantially obviate one or more of the 
problems caused by the limitations and disadvantages 
of the related art. 

Features and advantages of the present 
invention will be set forth in the description which 

10 follows, and in part will become apparent from the 

description and the accompanying drawings, or may be 
learned by practice of the invention according to 
the teachings provided in the description. Objects 
as well as other features and advantages of the 

15 present invention will be realized and attained by a 
coding apparatus and a coding method particularly 
pointed out in the specification in such full, clear, 
concise, and exact terms as to enable a person 
having ordinary skill in the art to practice the 

20 invention. 

To achieve these and other advantages and 
in accordance with the purpose of the invention, as 
embodied and broadly described herein, the invention 
provides an apparatus for coding a binary image 

25 representing an object shape, the apparatus 

including an inferior symbol detecting unit which 
decides which one of binary zero and binary one is 
an inferior symbol that is of smaller occurrence 
within a given area of the binary image, a divided 

30 portion generating unit which divides a rectangular 
block of the given area into divided portions , a map 
information generating unit which generates map 
information for each one of the divided portions. 
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the map information indicating whether a 
corresponding one of the divided portions has the 
inferior symbol included therein, and a coding unit 
which encodes only the divided portions that have 
5 the inferior symbol included therein, wherein an 
identification of the inferior symbol, the map 
information, and the encoded divided portions are 
output from the apparatus. 

The coding apparatus as described above 

10 divides a binary rectangular block that includes 

object interior pixels and object exterior pixels, 
one of which is inferior to the other in terms of 
frequency of occurrence, and the divided portions 
are encoded only when there is an inferior symbol 

15 included therein, thereby achieving efficient coding 
of the binary image representing an object shape. 

Further, the coding apparatus as described 
above reduces the load of the coding process while 
avoiding an efficiency reduction caused by data 

20 division, thereby making possible the real-time 
coding of a large shape image such as an image 
having the size of an HDTV image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

25 Fig.l is a block diagram of an object- 

shape coding apparatus according to the present 
invention ; 

Fig. 2 is an illustrative drawing showing a 
macro block having both inferior symbols and 
30 superior symbols and divided in half in both the 
vertical direction and the horizontal direction; 

Fig. 3 is a block diagram of another 
embodiment of the object-shape coding apparatus 
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according to the present invention, which includes a 
unit that identifies an inferior symbol for each 
small block; 

Fig. 4 is an illustrative drawing showing 
5 an example of a bit pattern of a small block; 

Fig, 5 is an illustrative drawing showing a 
bit pattern of a small block in which pixels are 
circularly shifted in the horizontal direction 
relative to the bit pattern of Fig. 4; and 
10 Fig. 6 is an illustrative drawing showing 

pixel line map information having bits Ml through M8 
that correspond to the horizontal direction pixel 
lines LI through L8 of Fig. 5. 

15 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In the following, embodiments of the 
present invention will be described with reference 
to the accompanying drawings. 

Before engaging in the describing of 

20 embodiments, the principle of the present invention 
will be described briefly. 

In the description that follows, pixels 
that are either interior pixels within the object 
boundary or exterior pixels outside the object 

25 boundary are defined as inferior symbols if they are 
those of smaller occurrence between the two types of 
pixels within the macro block, whereas the pixels of 
greater occurrence between the two types of pixels 
are referred to as superior symbols. 

30 Pixels that represent an object shape have 

relatively high correlation therebetween, so that 
the inferior symbols or the superior symbols tend to 
be concentrated. Since the superior symbols are 
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defined as those outnumbering the inferior symbols, 
there may be a case in which all the data within a 
divided portion are superior symbols. In the 
present invention, data indicative of whether at 
5 least one inferior symbol exists within the divided 
portion or indicative of whether all the data within 
the divided portion are superior symbols is 
transferred as a structural representation. 
(Hereinafter, this data is referred to as map 

10 information.) Use of map information as a 

structural representation makes it possible to 
eliminate a need for coding a divided portion 
comprised only of the superior symbols. This 
achieves an improved coding efficiency. 

15 In the present invention, a macro block is 

divided into a plurality of smaller blocks according 
to a predetermined procedure, and the obtained 
smaller blocks are further divided into pixel lines 
(lines of pixels) in the horizontal or vertical 

20 direction, thereby providing two stage data division. 
At each stage of data division, map information for 
divided data is transferred. The second stage 
division is not performed if the divided data 
obtained at the first stage include only superior 

2 5 symbols. 

A coded word is assigned to a bit pattern 
of a divided pixel line by using a variable length 
coding. In the present invention, pixels are 
rearranged first according to a predetermined 
30 rearrangement procedure such as placing inferior 

symbols at the beginning of a pixel line, and, then, 
the rearranged bit pattern is coded and transferred. 
In this case, information about the data 
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rearrangement needs to be additionally transmitted. 
Since the rearrangement improves the efficiency of 
variable length coding, however, the overall coding 
efficiency is also improved. 

In this manner, the present invention 
divides shape data along with use of structural 
representations, and assigns coded words. Namely, 
data division provides a basis for a simplified 
coding process, and use of the structural 
representation achieves highly efficient coding. 

Fig.l is a block diagram of an object- 
shape coding apparatus according to the present 
invention . 

In Fig.l, the object-shape coding 
apparatus includes an inferior symbol detecting unit 
1, a small block generating unit 2, a block map 
detecting unit 3, a pixel-line direction checking 
unit 4, a pixel-line generating unit 5, an in-line- 
pixel rearranging unit 6, a pixel-line map coding 
unit 7, a pixel-line coding unit 8. 

In the following, operations of the 
object-shape coding apparatus will be described. 

As input signals to the coding apparatus, 
the inferior symbol detecting unit 1 receives a 
rectangular macro block which includes object 
interior pixels as well as object exterior pixels. 
The inferior symbol detecting unit 1 outputs a macro 
block bit pattern that represents an image by two 
statuses, i.e., the inferior symbol defined as that 
of lesser occurrence between the interior pixels and 
the superior pixels in the macro block and the 
superior symbol defined as that of greater 
occurrence in the macro block. Further, the 
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inferior symbol detecting unit 1 supplies 
information about the inferior symbol, which is 
transmitted along with coded data. 

Each pixel of the macro block input to the 
apparatus of the present invention is either an 
object interior pixel or an object exterior pixel. 
In the following description, blocks and pixel lines 
are described as being a bit pattern having two 
statuses, i.e., the inferior symbol status and the 
superior symbol status. 

The macro block bit pattern supplied from 
the inferior symbol detecting unit 1 is input to the 
small block generating unit 2, which outputs a 
plurality of small blocks of bit patterns generated 
by dividing the macro block. The present invention 
is not limited to a particular method of dividing a 
macro block into small blocks. As an example, as 
shown in Fig. 2, a macro block having both inferior 
symbols and superior symbols may be divided in half 
in both the vertical direction and the horizontal 
direction, thereby generating four small blocks. 

The plurality of small blocks of bit 
patterns supplied from the small block generating 
unit 2 are input to the block map detecting unit 3, 
which outputs block map information indicative of 
whether an inferior symbol is present in each small 
block. This block map information is transmitted 
along with coded data. If there is an inferior 
symbol in a given small block, the bit pattern of 
this given small block is also output from the block 
map detecting unit 3. 

Fig. 3 is a block diagram of another 
embodiment of the object-shape coding apparatus 
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according to the present invention which includes a 
unit that detects the inferior symbol for each small 
block . 

When the configuration of Fig. 3 is 
compared with the configuration of Fig.l, the 
configuration of Fig. 3 has a small-block-inferior- 
symbol detecting unit 9 newly added between the 
block map detecting unit 3 and the pixel-line 
direction checking unit 4. Other elements are 
identical to those shown in Fig.l. In Fig. 3, the 
same elements as those of Fig.l are referred by the 
same numerals, and a description thereof will be 
omitted . 

The small-block-inferior-symbol detecting 
unit 9 receives the bit pattern of a small block 
from the block map detecting unit 3, and outputs 
information about the inferior symbol of the small 
block and the bit pattern of the small block that is 
represented by two statuses, i.e., the inferior 
symbol of the small block and the superior symbol of 
the small block. These outputs are supplied to the 
pixel-line direction checking unit 4. 

In what follows, the operation of the 
small-block-inferior-symbol detecting unit 9 will be 
described with reference to the four small blocks 
shown in Fig . 2 . 

Small blocks Bl and B4 shown in Fig. 2 have 
an inferior symbol of the small block that is 
identical to the inferior symbol of the macro block. 
With respect to these two blocks, the small-block- 
inf erior-symboi detecting unit 9 thus outputs the 
bit pattern of the small block as it was received, 
without any change, together with the inferior 
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symbol information of the small block. In small 
block B3, the inferior symbol of the small block is 
the superior symbol of the macro block. With 
respect to the small block B3 , therefore, the small- 
block-inferior-symbol detecting unit 9 outputs the 
inferior symbol information of the small block, and 
outputs a bit pattern in which the inferior symbols 
and the superior symbols are switched within the 

small block B3 . 

The bit pattern of the small block, which 
is output from the block map detecting unit 3 in 
Fig.l or supplied from the small-block-inferior- 
symbol detecting unit 9, is input to the pixel-line 
direction checking unit 4. The pixel-line direction 
checking unit 4 outputs information indicative of a 
direction of a pixel line, i.e., either the vertical 
direction or the horizontal direction, which is 
transmitted together with the coded data. The 
information indicative of a pixel-line direction may 
be defined and provided for each macro block, or may 
be defined and provided for each small block. 

Controlling factors as to which one of the 
vertical direction or the horizontal direction is 
selected as a pixel-line direction are not limited 
to particular implementation in the present 
invention , In this embodiment , the controlling 
factor is the number of lines in which an inferior 
symbol is present, and the direction is selected 
such as to make the number of lines smaller than 
otherwise. In the case of a bit pattern of a small 
block as shown in Fig. 4, for example, there are five 
horizontal pixel lines that include an inferior 
symbol, whereas eight vertical pixel lines (all the 
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vertical lines) include an inferior symbol. The 
horizontal direction that has the smaller number of 
lines is thus selected. 

The bit pattern of the small block and the 
information indicative of a pixel-line direction are 
supplied from the pixel-line direction checking unit 
4 to the pixel-line generating unit 5. The pixel- 
line generating unit 5 outputs a plurality of pixel 
lines of bit patterns generated by dividing the 
small block into the pixel lines. The bit patterns 
of the pixel lines are supplied to the in-line-pixel 
rearranging unit 6, which outputs pixel 
rearrangement information and the bit patterns of 
pixel lines in which pixels are rearranged as 
specified by the pixel rearrangement information. 
The pixel rearrangement information is transmitted 
together with the coded data. The pixel 
rearrangement information may be defined and 
provided for each macro block, or may be defined and 
provided for each small block. 

In the following, the rearrangement 
operation of the in-line-pixel rearranging unit 6 
will be described . 

In this embodiment, a bit pattern of a 
pixel line is circularly shifted by the 
rearrangement processing, and the shift length is 
used as the rearrangement information. This aspect 
of the present invention will be described with 
reference to Fig. 4 and Fig. 5. 

For the sake of explanation, it is assumed 
that the bit pattern of the small block shown in 
Fig. 4 (the pixel lines as indicated as Yl through 
Y8) is input to the in-line-pixel rearranging unit 6. 
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The pixels lines in the horizontal direction are 
subjected to circular shifting to the left by two 
samples, so that the order of columns XI, X2 , X3 , X4 
X5, X6, X7, and X8 are changed to X3 , X4 , X5 , X6 , X7 
X8, XI, and X2 . This produces a bit pattern as 
shown in Fig. 5. Pixel lines LI through L8 in the 
horizontal direction shown in Fig. 5 respectively 
correspond to the pixel lines Yl through Y8 in the 
horizontal direction shown in Fig, 4. 

In Fig. 5, the inferior symbols concentrate 
on the left-hand side in each horizontal pixel lines 
In this embodiment, variable length codes are 
structured by assuming the situations in which 
inferior symbols concentrate near the beginning of 
each pixel line (i.e., the left portion of a 
horizontal pixel line or the top portion of a 
vertical pixel line) . If such rearrangement 
processing results in a decrease in the volume of 
codes generated by coding, the shift length "2" 
indicating the shift by 2 samples is transmitted as 
the rearrangement information together with the 
coded data. Further, the bit pattern of Fig. 5 after 
the rearrangement processing is supplied to the 
pixel-line map coding unit 7. 

If the intended rearrangement processing 
results in an increase in the volume of codes 
generated by coding, the rearrangement processing is 
not actually performed, and the shift length "0" is 
transmitted as the rearrangement information 
together with the coded data. Further, the bit 
pattern of Fig. 4 as it is as received as input 
signals is supplied to the pixel-line map coding 
unit 7 . 
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The pixel-line map coding unit 7 receives 
the pit patterns of bit lines from the in-line-pixel 
rearranging unit 6, and outputs pixel line map 
information that indicates pixel lines in which an 
inferior symbol is present. Further, the pixel-line 
map coding unit 7 outputs the bit pattern of a pixel 
line with respect to each pixel line that has an 
inferior symbol included therein. The pixel line 
map information is a series of bits that are 
provided as many as there are pixel lines, and 
indicate whether an inferior symbol is present in 
respective pixel lines. This information is coded 
by variable length codes such as Huffman codes. 

In what follows, the operation of the 
pixel-line map coding unit 7 according to this 
embodiment will be described. 

For the sake of explanation, it is assumed 
that the bit patterns of the horizontal direction 
pixel lines Ll through L8 as shown in Fig. 5 are 
supplied to the pixel-line map coding unit 7. Fig. 6 
shows pixel line map information having bits Ml 
through M8 that correspond to the horizontal 
direction pixel lines Ll through L8 of Fig. 5. In 
this embodiment, the bits Ml through M3 of Fig. 5 
indicate that the respective pixel lines Ll through 
L3 include only superior symbols, and the bits M4 
through MB of Fig6 indicate that the respective 
pixel lines L4 through LB each include at least one 
inferior symbol. In this manner, the pixel-line map 
coding unit 7 according to this embodiment outputs 
the pixel line map information as shown in Fig. 6 and 
the bit patterns of the horizontal pixel lines L4 
through LB as shown in Fig. 5 in which at least one 
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inferior symbol is present. 

The pixel line map information and the bit 
patterns of horizontal pixel lines having an 
inferior symbol included therein are input to the 
pixel-line coding unit 8. The pixel-line coding 
unit 8 assigns codes to the bit patterns, and the 
obtained coded data is transmitted to a decoder side. 

As described above, the apparatus of this 
embodiment receives the small block of Fig. 4 
representing a partial object shape, and assigns 
codes to the line bit patterns M4 through M8 of 
Fig. 6 . 

According to the present invention 
described above, a binary rectangular block in which 
object interior pixels and object exterior pixels 
are both present are easily and efficiently coded. 

Further, the present invention reduces the 
load of the coding process while avoiding an 
efficiency reduction caused by data division, 
thereby making possible the real-time coding of a 
large shape image such as an image having the size 
of an HDTV image. 

Further, the present invention is not 
limited to these embodiments, but various variations 
and modifications may be made without departing from 
the scope of the present invention. 

The present application is based on 
Japanese priority application No. 2000-322696 filed 
on October 23, 2000, with the Japanese Patent Office, 
the entire contents of which are hereby incorporated 
by reference. 



